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ABSTRACT 

The intent of the investigation reported here was to 
study the impact of the Education Consolidation and Improvement Act 
(ECIA) on educational evaluation at the state and local level. The 

study had three major purposes: (1) to investigate the state 

education agency (SEA) and local education agency (LEA) responses to 
changing evaluation requi rements under Chapter 1 and Chapter 2 of 
ECIA, (2) to discover new evaluat ion-related methodological problems ^ 
arid (3) to identify changing evaluat ion training and technical 
assistance needs. Through a 10-month , longitudinal interview study of 
evaluators in five western state departments and five large school 
districts , some preliminary results emerged. Reductions in money and 
staff have indeed brought about consolidation , but improvement is 
still an open question. Budget reductions are affecting educational 
programs more than the new Chapter 1 and Chapter 2 legislation . Much 
of the Chapter 2 money is being spent on materials and equipment, 
especially microcomputers . Evaluation units seem to be shifting away 
from monitoring arid evaluation and focusing more on technical 
assistance arid leadership. These and other findings are discussed in 
this report. (BW) 
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PREFACE 



Tht Research on Evaluation Progr^ is a Northwest Regional 

Educational Laboratory project of research, development, testing, 
and training designed to create new evaluation methodologies for 
use in education. This document is one of a series of papers and 
reports produced by program staff, visiting scholars, adjunct 
scholars, and project collaborators — all members of a cooperative 
network of colleagues working on the development of new 
methodologies. 

What effect has the Education Consolidation and Improvement Act 
of 1981 had on evaluation practice at the state and local level? 
Through a ten month, longitudinal interview study of evaluators 
in five western state departments and five large school 
districts, some preliminary results begin to emerge; 

• Reductions in money and staff have indeed brought about 
consolidation* but improvement is still an open question. 

• Budget reductions are affecting educational programs more 
than the new Chapter 1 and Chapter 2 legislation. 

• Much of the Chapter 2 money is being spent on materials 
and equipment, especially microcomputers. 

• Evaluation units seem to be shifting away from monitoring 
and evaluation and focusing more on technical assistance 
and leadership. 

These and other findings are discussed in this report. 

Nick L. Smith, Editor 
Paper and Report Series 
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A STUDY IN CONTRASTS: EFFECTS OF THE EDUCATION 
CONSOLIDATION AND IMPROVEMENT ACT OF 1981 
ON SEA AND LEA EVALUATION 

Study Over view 

Ronald Reagan entered the White House two years ago with 
a commitment to overhaul the federal role in American 
education. 

"Education," he told the country in a televised address, 
"is the principal responsibility of local school systems, 
teachers, parents, citizen boards and state 
governments . " . • • 

Two years later > as members of the Administration are the 
first to acknowledge, efforts to redefine and reshape the 
Federal role in education show mixed results. 

(The New York Times, November 14, 1982) 

One of the basic nioves by the Reagan Administration in its 
effort to change the federal role in education was to propose the 
consolidation of most categorical education programs into a few 
large block grants, ebngress, reacting to pressure from those 
representing the disadvantaged, the handicapped, and others, 
resisted this move. 

In the end, the major legislation for the disadvantaged 
(Title I of the Elementary and Secondary Education Act, ESEA) , 
the handicapped (PL 94-142, Education for all Handicapped 
Children Act) vocational and adult education (The Vocational 
Education Act) , and a few other programs (e.g., impact aid* 
student financial assistance) survived as separate entities. 
But, 3 0-4 0 previously categorical aid programs were combined into 
one large block grant. Included were such programs as basic 
skills education, metric education, consumer education, gifted 
and talented education, and programs providing funds for library 
resources, textbooks and instructional equipment, guidance and 
counseling, desegregation, and teacher training and in-service. 

The disadvantaged student legislation, Title I, and the block 
grant legislation were combined in Subtitle D of Title V of the 
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Omnibus Budget Reconciliation Act of 1981 (PL 97-35) as 
Chapters 1 and 2, respectively* of the Education Consolidation 
and Improvement Act of 1981 (ECIA) . The resultant effects of 
Chapter 1 and Chapter 2 offer graphic examples of the mixed 
results achieved by the Administration. 

The reasoning behind Chapter 1 and Chapter 2 is reflected in 
the legislation itself (PL 97-35, August 13, 1982) • The 
Chapter 1 Declaration of Policy states that it is the intent of 
the legislation to: 

eliminate burdensome, unnecessary, and unproductive paper 
work and free the schools of unnecessary Federal 
supervision, direction, and control. 

(p. 95 STAT. 464) 

In addition, it is intended that the legislation will do away with 

overly prescriptive regulations arid administrative 

burdens which are riot necessary for fiscal accountability 
and make no contribution to the instructional program. 

(p. 95 STAT. 464) 

The purpose of the block grant (Chapter 2) was clearly to 
free state arid local education agencies to set their own 
priorities and conduct programs as they saw fit. Beyond this 
intent, the new legislation was: 

to do so in a manner designed to greatly reduce the 
enormous administrative and paperwork burden imposed on 
schools at the expense of their ability to educate 
childrea. 

(p. 95 STAT. 469) 

Clearly, the intent of EC I A was to shift the re sponsibility 
for setting policy, monitoring fiscal procedures, arid designing, 
operating, and evaluating educational programs out of 
Washington, D.C. The full impact of this shift on state and 
local education agencies and ultimately on this nation's children 
is now only beginning to manifest itself. 



The intent of the investigation reported here was to study 
the impact of the ECIA legislation on educational evaluation at 
the state and local level. The study had three major purposes: 
(1) to investigate the state education agency (SEA) and local 
education agency (LEA) responses to changing evaluation 
requirements under Chapter 1 and Chapter 2 of ECIA, (2) to 
discover new evaluation-related methodological problems, and 
(3) to identify changing evaluation training and technical 
assistance needs. 

We were able to focus only on the initial impact of ECIA 
since, as of November 1982 when the final interviews for this 
study took place, Chapter 2 of this legislation had been in 
effect for five months and Chapter 1 for just two months. In 
some cases our preliminary findings may hold true for the long 
run. In other cases, additional changes may alter dramatically 
the perceptions and implementation of Chapter 1 or Chapter 2 
requirements. It is precisely because final results are not yet 
available, that these "early returns? are so important. They can 
provide us with initial insights into the impact of ECIA and can 
help to shape further research on this and other legislation. 

In order to fully understand the effect of Chapters 1 and 2 
on program evaluation operations, on evaluation methodology, and 
on evaluation training and technical assistance needs, we need to 
assess first the general impact of the legislation at both the 
SEA and ££ft levels. Therefore, we examined changes in the 
following four areas related to the impact of ECIA. 

• The evaluation contexts monetary and staff-related 
changes resulting from the enactment of ECIA. 

• The transition to Chapter 1 and Chapter 2: 
administrative, structural, and programmatic changes. 

• Shifts in evaluation and monitoring focus and 
responsibility; 

• Emerging evaluation information and technical 
assistance needs. 



These four areas form the framework for analyzing and reporting 

_ _* __ 

the information collected from our field respondents. Before 
summarizing that ihf ormatibn, however, it is necessary to review 
briefly the Chapter 1 and Chapter 2 legislation and the design of 
the study reported here. 

Background on Chapter 1 and Chapter 2 

We can appreciate better the impact of the ECIA on program 
evaluation and more clearly understand the responses of SEAs and 
LEAs if we first know the major features of the legislation and 
especially how it differs from previous legislation- For this 
purpose, brief summaries oZ Chapter 1 and Chapter 2 are presented 
next. 

Chapter 1 . The following is the Declaration of Policy as it 
appears in ECIA Chapter I: 

CHAPTER 1 — FINANCIAL ASSISTANCE TO MEET 
SPECIAL EDUCATIONAL NEEDS OF DISADVANTAGED CHILDREN 

Declaration of Policy 

Sec. 552. The Congress declares it to be the policy 
of the United States to continue tc provide financial 
assistance to State and local educational agencies to 
meet the special heeds of educationally deprived 
children, on the basis of entitlements calculated 
under Title I of the Elementary and Secondary Educa- 
tion Act of 1965, but to do so in a manner which wiill 
eliminate burdensome, unnecessary, and unproductive 
paperwork and free the schools of unnecessary Federal 
supervision, direction, and control; Further, the 
Congress recognizes the special educational needs of 
children of low^income families, and that concentra- 
tions of such children in local educational agencies 
adversely affect their ability to provide educational 
programs which will meet the needs of such children. 
The Congress also finds that Federal assistance for 
this purpose will be more effective if education 
officials, principals, teachers, and supporting 
personnel are freed from overly pre script ive regula- 
tions and administrative burdens which are not 
necessary for fiscal accountabiity and make no 
contribution to the instructional program. 

(PL 97-35, August 13, 1982, p. 95 STAT. 464) 
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The specific changes embodied in Chapter 1 are as follows: 



• a reduction in the percent allowed for state 
administration of these programs from 1.5% to 1%; 

• the elimination of all requirements for Parent 
Advisory Councils; 

• a reduction in the requirement for maintenance of 
effort from 100% to 90% of the second preceding 
year for either aggregate expenditures or for 
combined fiscal effort per student; 

• the transfer of responsibility for waivers of the 
maintenance of effort requirement from the federal 
government to the State Education Agency; 



• the elimination of requirements for a 
comparability report; 

• a ban on the issuance of regulations by the 
Secretary of Education in the areas of program 
planning , development, implementation* or 
evaluation; and 

• a statement that* "Regulations issued pursuant to 
this subtitle shall not have the standing of a 
Federal, statute for the purposes of judicial 
review." 

(American Institutes for Research, 1981, p. iv. ) 1 

Since the specific focus on this study is program evaluation at 
the state and local level, the sixth item on the above list is of 
special interest. In essence, this ban on the issuance of 
regulations regarding evaluation as well as other related topics 
means (1) that the Title I evaluation models (A — norm referenced, 
B--control group and C— ^criterion referenced) are ho longer 
required, (2) that any evaluation schedule which satisfies the 
frequency requirement of "at least once every three years" may be 
used and (3) that any objective measurement instrument may be 
used. 

While these changes are dramatic in terms of their potential 
impact on Chapter 1 program operation and evaluation, they are 
modest compared to the sweeping changes in program and evaluation 
focus embodied in Chapter 2. 
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Chapter 2 . The following is the statement of purpose as it 
appears in EC I A Chapter 2: 



CHAPTER 2— CONSOLIDATION OF FEDERAL PROGRAMS 
FOR ELEMENTARY AND SECONDARY EDUCATION 



Statement of Purpose 

Sec. 561. (a) It is the purpose of this chapter to 
consolidate the program authorizations contained in — 

(1) titles II, III, IV, V, VI, VIII, and IX 
(except Part C) of the Elementary and Secondary 
Education Act of 1965 

(2) the Alcohol and Drug Abuse Education Act; 

(3) part A and section 532 of title V of the 
Higher Education Act of 1965; 

(4) the Follow Through Act (on a phased basis); 

(5) section 3(a) (1) of the National Science 
Foundation Act of 1950 relating to i lecollege 
science teacher training; and 

(6) the Career Education Incentive Act; 

into a single authorization of grants to States for 
the same purposes set forth in the provisions of law 

specified in this sentence, but to be used in 

accordance with the educational needs and priorities 
of State and local educational agencies as determined 
by such agencies. It is the further purpose and 
intent of Congress to financially assist State and 
local educational agencies to improve elementary and 
secondary education (including preschool education) 
for children attending both public and private 
schools, and to do so in a manner designed to greatly 
reduce the enormous administrative and paperwork 
burden imposed on schools at the expense of their 
ability to educate children; 

(b) The basic responsibility for the administration 
of funds made available under this chapter is in the 
State educational agencies, but it is the intent of 
Congress that this responsibility be carried out with 
a minimum of paperwork and that the responsibility for 
the design and implementation of programs assisted 
under the chapter shall be mainly that of local 
educational agencies, school superintendents and 

principals, and classroom teachers and supporting 

personnel, because they have the most direct contact 
with students and are most directly responsible to 
parents. 

(PL 97-3 5, August 13, 1982, p. 95 STAT. 469) 
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There appears to be some disagreement as to the exact number 
of programs "folded into" Chapter 2. The law itself does not 
give a number but simply lists where the programs are 
consolidated from. In the final regulations for ECIA it is 
stated that, Chapter 2 consolidates 28 programs into a single 
grant (Federal Register, November 19, 1982). A fact sheet oh the 
authorization bill numbers them as 29, but in the accompanying 
list there are only 27 programs named. Keating (n.d.) notes that 
ECIA "repealed or combined up to 42 categorical programs. . .with 
Chapter 2 funding the remaining 29." (p. 1) 

Hansen (1981) states that, "the list of authorized activities 
is very general and quite extensive apparently including all of 
the activities contained in ■antecedent legislation 1 ", that is, 
"the 32 previous programs which are being ■folded into* this one 
title." (p. 4) A count of the topics and subtopics under the 
"authorized activities" of Chapter 2's Subchapter ft, B, and C 
results in a list of 33 activities (PL 97-35, August 13, 1981, 
p. 95 STAT. 473-476) . An LEA may choose to spend all of Its 
Chapter 2 funds oh one activity or it may spread the funds among 
any combination of activities. The authorized activities are 
summarized in Appendix A of this report. 

As will be seen in the results section of this report, some 
states and school districts are confused and concerned about the 
long and sometimes vague list of "authorized activities." As 
Keating (n.d. ) notes: 

The problem of competition within the district is 
intensified by the virtual absence of federal technical 
assistance with respect to the consolidated programs 1 
goals of implementation. Thus those who have already 
received federal support will be in a much better 
position to secure their programs. • • 

(p. 4) 

In addition, Chapter 2 evaluation requirements are virtually 
nonexistent. State and local applications must only contain 
agreements to keep such records and provide such information as 
reasonably "may be required for fiscal audit and program 
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evaluation, consistent with the responsibility of" the Secretary 
and state. (PL 97-35, August 13, 1981, Sections 564(a) (6), and 
566(a) (3j) In addition, according to Section 564(a) (5), a 
state must submit an application which: 

beginning with fiscal year 1984, provides for an annual 
evaluation of the effectiveness of programs assisted 
under this chapter, which shall include comments of the 
advisory committee, and shall be made available to the 
public. 

(PL 97-35, August 13, 1981, p. 95 STAT. 471)) 

This is a far cry from the specific evaluation requirements of 
previous legislation (eigi, Title IV-C ESEA Improvement in Local 
Practices, Title IV ESEA Emergency School Aid for School 
Districts Undergoing Desegregation, Title III ESEA Special 
Projects) . 

Thus under both Chapter 1 and Chapter 2 we see a dramatic 
lessening of the previous federally imposed mandate to evaluate 
educational programs. 

Study Design 

Five western states were chosen for this study as being 
representative of states in this part of the country. They, 
therefore, provide only a limited sample of perspectives 
regarding the impact of ECIA. Western states were selected 
because an explicit focus of this study was the area of emerging 
information and technical assistance needs of states served by 
our agency. Within these states, people were interviewed who 
were (a) responsible for evaluation in general and/or (b) 
responsible for Chapter 1 and Chapter 2 programs in particular, 
at the state level and in the largest school district in each 
state. 

Six 15- to 30-minute telephone interviews were conducted with 
each field re spondent from February to November 1982. 
Respondents were interviewed once a month from February to June, 
and then, a final interview was conducted in November. The 
benefit of the telephone interview technique was that it could 
easily fit into respondents' schedules and did not necessitate 
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any travel on the part of the interviewer; ft limitation was that 
some of the rapport of face-to^face contact may have been lost. 

A semi-structured interview instrument was developed and 
revised on the basis of pilot trials. Subsequent modifications 
were made to the form as necessary to reflect changing field 
conditions. The same basic questions were addressed throughout 
the study, however, and form the basis of the results presented 
here. 

At the time of the initial interviews, the EC I A was not yet 
in effect, and even in November there had been a. maximum of five 
months of experience with it. However, our respondents knew the 
thrust of the Act and were preparing for its implementation well 
in advance of its starting date. It was, therefore, possible to 
follow changes in anticipated effects of the Act up to the point 
of its formal beginning and then to compare those expectations 
with perceptions formed during the actual operation of the Act. 
Also, in the final November interview, respondentia were asked to 
send backup documentation concerning Chapter 2 allocations and 
evaluation plans. This documentation provided a rich source of 
information for illustrating the changes which have occurred 
relative to Chapter 2. 

The Evaluation Context 

The aim of this study was to capture the way professional 
educators are responding to the evaluation related aspects of the 
Education Consolidation and Improvement Act's (ECIA) Chapters 1 
and 2. In or£er to view their responses from the proper 
perspective, it is necessary to know (1) with whom we talked, and 
(2) what changes have occurred in their funding and staffing 
levels. This first area of ECIA impact is discussed below. 

Respondents 

Over the course of this study we talked with more than 30 
professional educators at the state and local level. At the 
state level there was a change of personnel in three of the five 
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statewide evaluation positions and one of the Chapter 1 
positions. One state experienced two changes in evaluation unit 
directors in the 10-month period of the study. At the local 
level there were three changes in personnel. That meant we 
talked with new people in one district-wide evaluation unit and 
two Chapter 1 units. 

There were 24 respondents to the final interview in November: 

5 SEA evaluation directors 
5 LEA evaluation directors 
5 SEA Chapter 1 directors 
5 LEA Chapter 1 directors 
4 LEA Chapter 2 sources 

In relation to Chapter 1 and Chapter 2 evaluation, SEA evaluation 
directors indicated the following pattern of focus: 3 - Chapter 
2 only; 2 - both Chapters 1 and 2. Three of the five LEA 
evaluation directors reported responsibility for both Chapter 1 
and Chapter 2 evaluation. Two had no Chapter 2 responsibility, 
but only Chapter 1. Of course, the directors and their staff had 
other responsibilities, such as statewide and district-wide 
testing programs, research and other evaluation studies, policy 
analysis> and technical assistance to other state and local 
units. In fact, Chapter 1 and Chapter 2 evaluation was a 
relatively small part of their total responsibility. 

As would be expected, SEA and LEA Chapter 1 directors 
reported responsibility primarily for Chapter 1 and related 
evaluation. For example, besides responsibility for processing 
Chapter 1 data, they reported responsibility for other state 
compensatory education programs, including bilingual, Indian, 
early childhood, arid neglected arid delinquent education programs 
as well as responsibility for district-wide testing. One SEA 
Chapter 1 director also reported Chapter 2 evaluation 
responsibilities. 

Because of the newness of Chapter 2, and in many cases the 
lack of well defined evaluation procedures, four local directors 
recommended that four new Chapter 2 sources be interviewed. 
These sources were sometimes in curriculum units and other times 
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in grants management- type units. They were invaluable in 
providing detailed information about Chapter 2 expenditures and 
evaluation plans. 

Cha ng in Funding and St af £ 

It became clear by the November interview that most of the 
agencies surveyed had sustained some budget cuts and staff 
reductions over the last year. In some cases the reductions had 
begun as early as two years ago due to economic and local 
political changes. Some reductions in the last year took place 
in anticipation of 1982-1983 cuts. In one case a small, fifteen 
percent, cut had occurred this year and a similar reduction was 
anticipated in each of the next few years. This would gradually 
reduce staff. In another case, large cuts were taken in travel 
and materials allocations in order to save staff positions. 

Data from state *?nd local evaluation directors show that five 
reported changes in budgets and four reported no change- While 
no direct data were available from the tenth agency, the general 
reorganization currently underway and the related staff reduction 
already experienced by other units suggests at least modest 
reductions. 

1981-1982 Chapter 1 funding from the federal government was 
reduced by 20-30% in comparison with 1981-1982 Title I funding. 
There are no specific figures available to compare Chapter 2 
funds with the total of all the funds that were allocated for the 
various categorical programs that it subsumed. However, all of 
bur respondents who were knowledgeable about the categorical 
programs and Chapter 2 noted serious reductions. 

Reductions in funds were experienced not only relative to 
federal sources but also state sources. This was due to the 
general economic troubles of the states and cities in our 
sample. As a result of funding reductions, three state Chapter 1 
respondents reported staff reductions ranging from 25-33%. One 
local Chapter 1 respondent reported a 50% cut in staff. One 
state evaluation director reported a 75% reduction in staff, 
while three local level evaluation directors reported staff 
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reduction (1, less than 5%, and 2, from 55-80%). All together, 
eight of the twenty respondents reported some reductions in staff. 

The reported number of budget reductions or staff reductions, 
or lack of reductions, does not, however, present the whole 
picture of changes at the state and local level. The absence of 
much change in some states has to do with the fact that they were 
already receiving the minimum state allocation and that has not 
changed. However, larger states have had pronounced cutbacks. 

The federal contribution to any state or local evaluation 
budget is generally modest. In the February interviews, SEA 
evaluation directors reported from 0% to 59% when asked about the 
federal share of their evaluation budget (the average was 30%) . 
Local evaluation directors reported a 10% to 18% federal share in 
their evaluation budgets (the average was 15%) . Eighteen percent 
of the total evaluation office budget was the figure reported as 
the average federal share of a national sample of 215 local 
evaluation offices in school districts with 10,000 or more 
students (Lyon et al. , 1978, p. 57). 

The word "consolidation" , as in Education Consolidation arid 
Improvement Act, is especially apt in talking of Chapter 2 budget 
and staff changes. In the past, people have been funded from a 
variety of sources and have, as a result, performed a variety of 
duties. Now there has been a consolidation. The comments of one 
SEA evaluation director help to illustrate this changes 

[the] change that has taken place is that [whereas] 
previously some of bur employees were funded with bits 
and pieces of administrative budgets of programs they 
were covered in, now everyone in this section is being 
funded by the 20% of Chapter 2 funds earmarked for use at 
the state level. 

What we have seen is a consolidation of staff in terms of the 
organization* a modest reduction in terms of numbers, and a 
subsequent broadening of responsibilities. The potential result 
of these changes is that staff will have less time to spend with 
their clients and, therefore, their contact will be more 
superficial than before. This change in the nature of the 
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relationship between SEA evaluators {both Chapter 1 and 
Chapter 2) and their clients is described in terms of a move away 
from program evaluation and thorough program monitoring to more 
leadership, technical assistance, and cost accounting as 
monitoring • 

The specific changes embodied in Chapter 1 and Chapter 2 are 
discussed in the next section. 



Before looking at the specific evaluation related effects of 
EC1A, it might be well to examine the important general changes 
which occurred in the transition to ECIA. This is the second 
area of focus of this study. 

Plans for implementing Chapter 1 and Chapter 2 were at quite 
different stages in February. In many cases these differences 
continued into November. The differences were the consequences 
of the nature and extent of the changes embodied in Chapter 1 and 
Chapter 2. Understanding these changes will help to make clear 
the reasons behind the mixed results of the Education 
Consolidation and Improvement Act. 

K4nd s of Change 

There are essentially five kinds of changes that are either 
the intent and/or the result of ECIA: 

• organizational changes— new lines of communication 
(administrative reorganization) 

• structural change — new areas of responsibility 
(functional reorganization) 

• program justification change — new reasons for 
offering particular programs 

* 

• program operation change — new way of running a 
program 

• program effect change — new outcomes of a program 

Our preliminary findings suggest that the ultimate effects of 
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Chapter 1 and Chapter 2 may not be consistent with their 
intents. While the intended effects of the acts have potential 
for influencing evaluation methodology and technical assistance 
and training needs, it is the results which must be examined most 
closely in order to assess the real impact of ECIA. 

Transleioo-^tc- Chapter 1 

Chapter 1 intent, as implied by its declaration of policy, 
seems to suggest organizational and structural change, since 
administrative and functional elements such as paperwork, 
unnecessary federal supervision, direction, and control, and 
overly prescriptive regulations and administrative burden are to 
be reduced or eliminated. Evident from the comments of state 
Chapter 1 respondents (formerly Title I respondents) is that the 
major components of Title I were transferred to Chapter 1. Let 
us look more closely at what changes did or did not occur 
regarding Chapter i; 

Organizational change - All five SEA Title I respondents said 
they were going to continue the central evaluation mechanism 
called Title I Evaluation arid Reporting System (TIERS), with 
minor modifications. This was confirmed in the November 
interviews. In May, four of the five state level Title I 
respondents reported having conducted workshops related to the 
new legislation with the fifth planning to provide a workshop for 
local districts in the fall. However, no surprises were 
anticipated; As of June, three of the five indicated they would 
use the same formula as last year to allocate funds to local 
districts. There were no data from the other two SEA Title I 
respondents on this point; 

The elimination of the requirement for Parent Advisory 
Councils is another change in the "lines of communication." In 
light of the shift of responsibility for rule-making from the 
federal to the state level* our respondents reported that they 
are retaining the spirit of parent involvement, but are viewing 
the organization of that involvement in a much more flexible, and 
they feel more effective, way. Some SEAs are changing the 
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requirements regarding the composition of parent advisory 
councils (PACs) . The PACS are becoming less representative* but, 
they feel/ more productive. That is, people are included not for 
just the sake of representation/ but because of their willingness 
to work and commitment to the program. Some I^As are eliminating 
school building councils and only having a district council. 
Others have established a Chapter 2 advisory council that draws 
part of its membership from the Chapter 1 council. This is a 
good example of the impact of ECIA in increasing the options of 
the local administrator. 

While there was much activity at the state level from 
February to June ir> getting ready organizationally for Chapter 1 
(e.g. , rewriting forms, informing people of major changes, 
working oh a non-binding handbook) there was almost no reported 
activity at the local level. 

Structural change. It is important to note that in early 
interviews, when state and local Title i directors were queried 
about such matters as level of paperwork, supervision, 
administrative burden and so on, four of the five SEA Title I 
directors said present paperwork was not burdensome, and one said 
it was burdensome but not unnecessary- Another respondent said, 
"a few reports require an excessive amount of paperwork, but in 
general most of the paperwork required is purposeful for a good 
educational program." Pour out of the five state Title I 
respondents anticipated little or no change in paperwork under 
Chapter 1. In November, four SEA Chapter 1 respondents reported 
a decrease in the amount of paperwork they were requiring of 
local districts. One reported no change in paperwork. 

Four of the five SEA Title I directors also said they had not 
suffered from unnecessary federal supervision or control. They 
anticipate somewhat to much less federal supervision under 
Chapter 1, but did not feel the local programs would sense any 
change. As one respondent noted, "Federal supervision has been 
indirect. The only change is that responsibility has now been 
formally passed on to state so locals won't see a difference." 
They also anticipated that state supervision will be less. 
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Three of five local Title 2 directors said in early 
interviews that they did hot perceive the paperwork as being 
unnecessary or burdensome, nor lid they suffer from unnecessary 
supervision or control. One respondent was not interviewed on 
this point and another reacted that the state reporting system 
was definitely burdensome and that there was unnecessary 
supervision. 

Three of the five local Title I directors also did not 
anticipate that burdensome and unnecessary paperwork would be 
eliminated under Chapter 1. The other two were simply not sure 
of the likely impact of Chapter 1 in this area. In the November 
interviews, four local Chapter 1 directors reported that even 
though the state was supposedly requiring less, they saw no 
change in the paperwork. One respondent did say that paperwork 
"hasn't been quite so monumental." 

Even though states eliminated some parts of the application 
and reporting process, for example, project descriptions, 
comparability descriptions, and parent advisory council data, the 
November interviews suggested that local administrators now felt 
pressure to maintain more records themselves in the event of an 
audit. In fact, some administrators saw OMB Circular A 102 
Attachment P as suggesting that audits might encompass both 
fiscal procedures and program performance. These concerns 
suggest structural changes at the local level that, in effect, at 
least maintained and in some cases increased paperwork and 
administrative burden- 
When interviewed in February, 1982, in most cases both SEA 
and LEA evaluation directors reported a wide range of evaluation 
responsibilities. These included, for example, evaluation of all 
federal and state categorical programs, operating the state or 
district assessment program, policy analysis, planning, high 
school graduation testing, research and evaluation design and 
proposal development, and needs assessment. 

Even Title I personnel reported a variety of other 
responsibilities. In addition to Title I evaluation and 
reporting, staff were responsible for evaluating bilingual 
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programs, Indian education, neglected and delinquent programs* 
and state driver education. They often had program leadership 
responsibility as well. This wide range of activities grew as a 
result of budget cuts and staff reductions as reported in 
November inter views. 

There was a consistent reporting of movement from monitoring 
and supervision toward leadership and technical assistance on the 
part of SEA evaluation and Chapter 1 respondents. It is not as 
though they did not do these before; it is just that with fewer 
staff and the loosening of Chapter 1 requirements, they were in a 
position to assume a role in which they felt comfortable. 

Both state and local respondents focused their attention oh 
local programs when asked about the extent to which the goals of 
flexibility, decentralization, and lessening of administrative 
burden have been achieved relative to Chapter 1. There was no 
discussion of the realization of these goals at the state level. 

Four out of five SEA Chapter 1 respondents said there was now 
more flexibility at the local level. But only two out of five 
LEA Chapter 1 respondents reported more flexibility. The other 
three reported there were no changes. Very few SEA respondents 
commented one way or the other about Chapter 1 related 
decentralization and lessening of administrative burden 
specifically. Those local directors who did see a change pointed 
oat that they have more flexibility in measurement and testing 
areas such as deciding which tests to use, when to test, and how 
to report results^ 

Overall, there Was not a strong or consistent feeling that 
there was much change in the structural aspects of Chapter 1 
concerning flexibility, decentralization, and administration. 

Program justification . None of our respondents made any 
comments about change in program justification. However, one 
local Chapter 1 e valuator did suggest that the need to reduce 
their program had caused them to target their program more 
carefully. 

The new requirements (listed in Appendix B of this report) 
essentially lay out the criteria for evaluating local programs. 
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The criteria include: {1) the requisite characteristics of 
students and areas within a district to whom services are to be 
provided; (2) the need to base services on "art artrtual assessment 
of educational needs"; (3) the requirement that programs be of 
sufficient "size, scope, and quality" to have the potential for 
success; (4) the necessity for program evaluation in terms of 
"objective measurement of educational achievement in basic 
skills" and whether "improved performance is sustained over a 
period of more than one year"; and finally (5) the requirement 
that provisions be made for serving "educationally deprived 
children attending private elementary and secondary schools." 

The reason for the lack of concern over program justification 
may be that the language in the Chapter 1 legislation regarding 
the characteristics which programs need in order to be authorized 
has simply been reduced in amount. The state and local Chapter 1 
directors have been living under similar requirements over the 
past few years. But they how are faced with less direction, an 
example of the freeing of local educators from "overly 
prescriptive regulations." It remains to be seen whether or not 
this will result in a significant change in the kinds of students 
served * the quality of programs provided, and the kinds of 
program evaluation carried out. 

Program impleme ntation . While there appears to be little in 
the way of intended program changes implied in the Chapter 1 
policy, it became apparent from the November interviews that 
there were some major changes in local programs due to budget 
reductions. 

Adjustments at the administrative level could be made by both 
SEAs and LEAs to compensate for budget cuts and inflation. One of 
the districts in the study had a 19% increase in teachers' 
salaries this year. However, since local program staff represent 
the bulk of program costs, staff were the major focus of cuts 
given reductions in the budget. 

An LEA Chapter 1 respondent reported that, to compensate for 
budget reductions, an entire elementary school was dropped from 
the Chapter 1 program. In another district it was reported that 
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the number of Chapter 1 teachers fell from 80 to 50. In another 
district, teacher aides became the focus of cuts, dropping from 
15 to i at the elementary level and from ii to 5 at the junior 
high level. 

Program effect . Program changes resulting from staff 
reductions and budget cuts have had a definite effect on the 
provision of services to Chapter 1 students. For example, the 
exchange between the interviewer (Q) and one respondent (A) went 
like this: 

Q. What would you say the impact of losing almost 
one million dollars has been on your Chapter 1 program? 

A. It has been separated on two issues. Revised 
Chapter 1 regulations have seen very little change in 
the operation of Title I. _ The real change has come 
about. because of decreased funding, which has severely 
crippled bur program. I was at a school this morning 
that had five aides and a teacher last year; this year 
it has one aide and a half-time teacher. That's a big 
difference in numbers of kids they can serve, etc. 
They've reduced their math services to working with 10 
kids as opposed to 40. That resulted because of 
budgetary cuts, not Chapter 1 regulation. 



Q. What would you say is the decrease in the 
number of children served? 

A. Probably 1,000— v?e s re down to 8,000 from J, 000, 
approximately. When you go to individual schools you 
can really see what that means... 

As teachers and aides are eliminated, the number of students 
served necessarily drops since state guidelines specify the ratio 
of staff to students served. The effects on students implied by 
this resjponderit f s comments seem to run counter to the intent of 
Chapter 1 to maintain services to children. 

General conclusions and the implications for further research 
of the Chapter 1 organizational, structural and programmatic 
changes are discussed in the last section of this report. 
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Summary of preliminary Chapter 1 findings. A brief overview 
of Chapter 1 results is presented next. These preliminary 
findings are based on the discussion in the previous sections of 
this report. This discussion focuses oh the sometimes subtle 
differences between state and local perceptions and provides 
examples of differing perceptions in regard to such concepts as 
paperwork, supervision, and flexibility. This discussion should 
be kept in mind when reading the following findings. 

The Title I Evaluation and Reporting System (TIERS) is 
in essence being retained in all states represented in 
this study. 

Parent involvement in Chapter 1 programs is taking on 
new forms; 

Title I paperwork was not seen as burdensome and 
unnecessary, nor has any change in this regard been 
reported at the local level relative to Chapter 1 
paperwork; 

There was, overall, little perceived change in 
flexibility, decentralization or administrative burden 
reported by state and local Chapter 1 directors. 

Chapter 1 budget reductions have had a dramatic impact 
at the local level resulting in cuts in instructional 
staff and consequently a reduction in the number of 
students served. 

Transition to Chapter 2 

Sweeping organizational^ structural, and programmatic changes 
are implied by the Chapter 2 Statement of Purpose, Program 
priorities are to be decided by state and local agencies; "the 
enormous administrative and paperwork burden" is to be "greatly 
reduced," and the responsibility for the design and 
implementation of programs "shall be mainly that of local 
education agencies. . .because they have the most direct contact 
with students and are mostly responsible to parents." (PL 97-35, 
95 STAT. 469, August 13, 1982). 

Here* again, the changes that resulted are hot altogether 
those implied in ECIA f s stated purpose. Based on our interviews, 
there appears to have been a great deal of Chapter 2 related 
activity from February to June at the state level, and as of 
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November, at both the state and local level. In order to get a 
more complete picture of Chapter 2 changes, four respondents were 
added to the local evaluation group for the November interviews. 

Our respondents reported that state activity began as early 
as December 1981 in one state, with the establishment of the 
required State Advisory Committee. As of May 1982 these 
committees had recommended allocation formulas, put out reports 
and made policy recommendations, and, in general, met their 
statuatory requirements. In at least one case their 
recommendations focused quite specifically on program evaluation 
matters. For example, one respondent said: 

As far as evaluation goes, we have an interesting 
phenomena in that the state advisory group is trying 
to require more than the statute will permit them to 
require, and we are working with them to get some 
sensible middle ground... 

Other activities took place in regard to state level 
preparation for Chapter 2. There was much general planning going 
on from February to June. In addition, some states developed: 

• state level priorities 

• forms for local districts to use to apply for Chapter 2 
funds 

• plans for monitoring and evaluating Chapter 2 programs 

• computer programs to account for Chapter 2 funds 

Many SEAs also offered workshops to inform local school districts 
of Chapter 2 changes. 

Just what changes have occurred relative to Chapter 2? Next 
we take a look at each of the five areas, of change from both the 
local and state perspective to see what has and has not changed. 

Organizational change . Due to the consolidation necessitated 
by reduced budgets and staff and by the block grant structure, 
there are new lines of communication resulting from 
reorganizations at both the state and local levels. Generally, 
each LEA evaluation unit has experienced a budget cut which has 
resulted in a change in staffing. A change in staffing has meant 
a consolidation of offices and responsibilities. This has 
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affected the scope of research and evaluation at the LEA level. 
Many of these changes began in early 1982, most likely in 
anticipation of new federal legislation and because of the 
recession throughout the western part of the United States; 
Larger LEAs tended to experience greater budget cuts than smaller 
LEAs. Along with the budget cutis came staff reductions and 
internal reorganizations. 

Here, again, consolidation was the watchword. Instead of 
multiple program administrators scattered throughout the 
organization, many SEAs and LEAs are consolidating Chapter 2 
staff and forming more integrated units. 

Structural change . There seem to have been some important 
changes in regard to Chapter 2 responsibilities. A shift has 
occurred away from monitoring and evaluation toward technical 
assistance and leadership. While these services do represent the 
beginning of a new emphasis, they were all part of staff's former 
role. 

A new group to emerge is the Chapter 2 Advisory Committee. 
All states and at least orie district used advisory committees for 
such tasks as identifying areas of program emphasis, developing 
allocation formulas, and suggesting evaluation requirements. 

There is a clear and consistent perception among LEA 
respondents regarding the effects of Chapter 2 vis-a-vis 
flexibility, decentralization, and administrative burden. All 
LEA Chapter 2 respondents agreed that Chapter 2 has provided both 
flexibility and decentralization in terms of deciding how and 
where money should be used. For example, they felt that they now 
have the flexibility to meet the needs of particular schools and 
students. One SEA respondent noted the "effect is pretty much 
what the intent was: decentralization, flexibility, and 
lessening of administrative burden have been essentially 
accomplished." Three out of the four SEA evaluation directors 
responding to this point also agree that there was increased 
flexibility and decentralization. 
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However , there is a mixed reaction regarding administrative 
burden. Some respondents view less paperwork and fewer general 
regulations as lessening administrative burden. For others this 
means more responsibility for decision making , planning , 
coordination and monitoring. One respondent said, "There's 
probably more administrative burden passed on to the local level 
than we had under the programs of Chapter 2, inasmuch as someone 
has to mind the store*.. " Still others view less paperwork as a 
potential threat to the long-term maintenance of a program. As 
one local level respondent said, "down the road a piece, •• .when 
it comes time for reauthorization and you look for good data, you 
won't have it, and that's going to turn around and bite us." An 
SEA respondent said "possibly we're asking for them to do less 
this year, but I think that they're (the Feds) going to clamp 
down on us so we had better have the information." 

Program justification . At the state level there is an 
obvious change in the basis for program justification since 
requests for program funds no longer have to be justified 
categorically. Instead, SEA respondents agreed, whatever program 
areas were identified by LEAs would have to be honored by the 
state. Some stated did, however, establish priority areas within 
the three subchapters of Chapter 2 through the use of their 20% 
set aside. (See Appendix A for the areas included under 
Chapter 2 Subchapters A, B, and C. ) 

With input from the advisory committee, for example, one 
state designated various areas as targets for funding. They 
included: expanding the assessment program, secondary math and 
English curriculum development and implementation, common 
requirements for high school graduation, and staff development 
centers with the major thrust of microcomputer curriculum. 

A variety of means were used to determine Chapter 2 
expenditures -\t the local level. One district used an advisory 
committee to decide which programs to fund. The respondent from 
this district told us: 
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We involved staff, we had superintendents, 
administrators, teacher organization [representatives] , 
a number of parents from our citizens^ 5f v if w 
committees. They all reviewed program priorities and 
anticipated district priorities in the future and we 
elected to go with a few of those projects from the 
past, and then we started a couple of new ones; also 
eliminated a couple from the pasti Essentially it was 
based on recommendations of the different groups that 
were involved^ 

In another case, the new statewide adoption of reading and math 
materials meant that most of the local money would be going into 
textbooks and related material. 

In one district a unique basis for program justification was 
reported. This district experienced a dramatic cut in its 
federal funds over the last few years. This year alone its 
desegregation funds dropped from 11 million to one million. In 
order to support its desegregation effort, all of its Chapter 2 
funds are earmarked for this program (Subchapter B, Number 8) . 

Programs in individual schools in this district are therefore 
justified on the basis of their contribution to the district 
desegregation plan. That is, a program must have the potential 
to attract and hold the right proportion of students and should 
not be likely to adversely affect the basic skills achievement of 
students. The impact of such an approach on program 
implementation arid effects, as well as its implications for 
program evaluation and technical assistance and training, will be 
discussed in subsequent sections of this report. 

We did not talk to any small school district personnel, but 
as one respondent said, "If you added up air of the Chapter 2 
dollars, and all of the categorical dollars, we're [the large 
district] getting less; on the other hand, if you look at 
categorical program funds that Wert, distributed across the state, 
most districts today are getting more money than they did under 
categorical programs." Typically these new dollars ranged from 
$800 to $2,500 per district. 




We also found that, since there was a general uncertainty as 
to the future levels of funding, local administrators did not 
want to begin building new programs and adding staff. They felt 
much more comfortable buying things. 

Program-Implementation . Almost without exception, the areas 
that received the lion's share of Chapter 2 funds, typically 80% 
or more, are within the Subchapter B areas providing for "library 
resources, textbooks, and printed and published materials for use 
by teachers and students for instructional purposes" and other 
"instructional equipment and materials" (i.e., microcomputers). 
It is interesting to note that these kinds of expenditures give 
administrators a great deal of flexibility^ may be decentralized 
to the school building level, and reduce administrative burden 
because of the straightforward accountability procedures 
demanded. They are also highly favored by teachers since they 
provide them with the tools for teaching. 

Microcomputers were reported as being high on most districts' 
priority list. In one state this amounted to $794,000 being 
spent on what has been nicknamed "the Apple a day program." 
However, the extent and the rapidity with which they are being 
purchased also poses some serious questions at the policy, 
program, and evaluation levels. 

Beyond materials and equipment, as anticipated by Keating, 
those programs which were already in existance and which had an 
influential constituency and/or established staff and 
administration remained intact. The desegretation case is the 
most extreme example of this. 

Chapter 2 has also prompted some LBAs to adopt new and 
innovative practices. Two districts have developed their own 
mini-grant programs similar to the bid Title IV-C program 
designed to improve school practices. One state is also 
continuing a teacher incentive program. 

In all of these programs, teachers apply for grants via a 
proposal which must contain program objectives, procedures, 
staff, materials and an evaluation plan. Often principals are 
required to sign off on the proposal. The grants typically range 
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from $1,000 to $3,000 and have stimulated proposals in areas such 
as music, art, physical education, and computer software. In one 
district, six Title IV-C validated programs have been offered for 
adoption. Mini-grants are a good example of another kind of 
flexibility appreciated by local administrators. 

Table la and lb show the distribution of programs under 
Subchapters A, B, and C of Chapter 2 as reported by SEA and LEA 
respondents, respectively. People were naturally cautious in 
their implementation of programs funded with Chapter 2 money. 
The exception to this may be the purchasing of microcomputers. 
On the surface, they appear to be simply investments in 
equipment. However, in order for these investments to pay off, 
additional people and programs will have to be added. 

If funding becomes stable and there is not a proliferation of 
regulations, we may see a significant change in programs with the 
mini-grant and local advisory committee concepts leading the way- 

Program effect . There is an amazing diversity of areas of 
program effects which may result from the distribution of 
Chapter 2 funds. Microcomputer hardware arid software purchases, 
desegregation programs, and mini-grants are but the most obvious 
when thinking about the impact of Chapter 2. The effects of 
major purchases of library resources arid textbooks to the 
exclusion of other program expenditures is another interesting 
area. The impact of setting educational proficiency standards 
and providing staff development to improve instruction in basic 
skills is likely to be immense. 

Our respondents did not report any specific effects that they 
are anticipating. Therefore in the third section of this report, 
the policy, practice, and outcome questions stimulated by 
Chapter 1 and Chapter 2 changes will be discussed. 




Table 1 

Distribution of Chapter 2 Programs 
Among Subchapters A, B, and C 



Subchapter A 



Subchapter B 



Subchapter C 



SEA Report of Chapter 2 Program Distribution 



* 3 Staff deyeiopnent to 

improve instruction 
in basic skills 

1 Diagnostic needs 
assessment 

1 Parent participation 
activities 

* 1 Student testing and 

program evaluation 

1 Instruction in 
reading 



* 5 Instructional equipment 

* 3 library resources and 

textbooks 

1 Staff development 

X Programs to improve 
educational practice 

1 Froblems of minority 
isolation and concen- 
tration 

1 Guidance and counseling 

X Management improvement 
programs 

* 1 Desecration programs 



* 3 Talented aha gifted 

* 2 Educational profi- 

ciency standards 

1 Community education 

1 Environmental educa- 
tion 

1 Health education 
1 Career education 
1 Arts education 



b. LEA Report of Chapter 2 Program Distribution 



None reported 



* 3 Instructional equip- 

ment 

* 2 Library resources and 

textbooks 

2 Teacher inservice 

* 2 Desegregation programs 

1 Math/science mini- 
grants 



2 Talented arid gifted 

1 Educational profi- 
ciency standards 

1 Community education 

1 Law related education 

1 Arts education 



* ■ Programs reported to receive the greatest proportion of funds. 

# - Number of respondents reporting this program (a duplicated count overall). 



27 



Summary of preliminary Chapter 2 findings. As With the 
Chapter i results, the reader is encouraged to view these 
preliminary findings in light of the more indepth discussion 
presented in the previous sections of this report. The following 
statements are meant only as a brief overview of this discussion. 

The large school districts in our study reported a 
reduction in funds under Chapter 2 in comparison with 
the funds received under the previous categorical aid 
programs. 

It was reported that small districts will receive an 
increase of $800 to $2,500 as a result of Chapter 2. 

Typically, 80 percent or more of Chapter 2 funds will 
go for instructional materials and equipment (largely 
for microcomputer hardware) • 

Those categorical programs already in existance arc. 
likely to take the remaining Chapter 2 funds. 

There was a marked increase in flexibility and 
decentralization perceived at the local level as a 
result of the Chapter 2 legislations 

Shifts in Evaluation and Monitoring 
Focus and Responsibility 

Preceding discussions have implied that, as a result of ECIA f 
a varisty of changes have occurred in the focus of and 
responsibility for evaluation and monitoring among state and 
local administrators. In general, since Chapter 1 requirements 
are simply less specific than those of Title I, few changes in 
evaluation and monitoring practice are anticipated. Certainly, 
the provisions of Chapter 2 represent a de-emphasis of program 
evaluation in comparison with previous categorical programs. In 
this section we discuss the shifts in evaluation and monitoring 
focus and responsibility as reported by each of four respondent 
groups. 

SEA Evaluation Directors 

State level evaluation directors reported in earlier 
interviews that they and their staff performed a variety of 
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evaluation and other tasks. As is evident from Table 2, these 
tasks cover Title 1 and other programs. They also entail a wide 
range of evaluation and monitoring responsibilities. 

There were no great changes in Chapter 1 evaluation and 
monitoring reported by directors of evaluation. As one 
respondent noted, "Title I is what they call an institutionalized 
program r It has a bureaucracy of its own, has a momentum of its 
own..." SEA evaluation directors indicated that they would 
continue to provide technical assistance, leadership, and 
monitoring support relative to Chapter 1. 

Depending on a state's use of Chapter 2 funds, that is, for 
programs or materials, the emphasis oh monitoring versus 
evaluation varied. For the purchase of instructional equipment, 
materials, library resources and the like, monitoring will be the 
primary focus. This will concern a rather straightforward 
accountability regarding "did you do what you said you were going 
to do?" 

SEA Chapter 2 respondents said they will conduct this type of 
monitoring in a variety of ways. One SEA evaluation re spondent 
indicated that he would undertake a simple analysis of the ways 
funds are being allocated, based on the application forms from 
each district. Districts will then be held accountable for 
documenting expenditures relative to their allocations. 

In another case, a computer file is being set up at the state 
level to track wher* the money went under the old multiple 
categorical grants. This information is to be compared and 
contrasted with where it is now going, especially in regard to 
the types of students who are being served. This will provide 
the basis of a system for monitoring districts' efforts to meet 
the needs of diverse students. 

Despite the lack of federal guidelines for evaluating 
Chapter 2 programs, a few SEAs are moving ahead with plans for 
requiring evaluations and for providing technical assistance to 
LEAs to help them develop unique evaluation designs, "unique in 
the sense of appropriate to the local situation." 
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Table 2 

Evaluation Topics and Other Tasks 
SEA Evaluation 

• All federal and state categorical programs 
a General assessment/state assessment 

• Planning 

• Task force assignments 

• Research and evaluation consultation 

• Needs assessment 

• Policy studies 

• High school graduation assessment 

• Title 1, migrant, PL 94-142, Title IV-B 

• Technical assistance 

• Traffic safety program 

• Computer support 

SEA Title I 

• Title I and state compensatory education 

programs, bilingual/bicultural programs 

• Assistance to LEA regarding special education 

• Neglected and delinquent education 

• Indian education 

• Migrant programs 

• State driver education program 

LEA Evaluation 

• Testing 

• Survey research 

• Technical assistance on test data 

• Needs assessment 

• Research screening and design 

• Proposal development 

• Financial planning 

• Reporting 

• Policy planning and analysis 

LEA Title I 

• Processing Title I data 

• Classroom consul tation/inser vice 

<o> Title I and Indian Program, Early Childhood 
Program, arid Disadvantaged Program 

• District-wide testing 
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Most anticipated they would be involved in Chapter 2 
evaluation "very minimally, in the first year." They anticipate 
playing roles of direction, support, and leadership. When they 
find "those [districts] that are doing something unique, we will 
make ah individual contact with them to talk about evaluation 
documentation." One respondent seems to summarize the situation 
very well: 

We don't require evaluation. We can encourage 
one at the local level and we'll have some districts 
that Will have some pretty valid information. 
Overall, on a statewide basis, our state, like other 
states, will be hard pressed to present some hard data 
indicating that this Program did what the BCIA 
indicates, educational improvement, particularly. 
Consolidation, yes — improvement is a question mark. 

Ail five SEA evaluation directors noted that, in applying for 
funds, an LEA is obligated to collect data for purposes of 
monitoring and evaluation. 

Summary of SEA evaluation director preliminary findings . 

Technical assistance, direction, support, monitoring 
and leader ship were the roles most often reported by 
SEA evaluation directors in relation to Chapter 1 
and/or Chapter 2 evaluation. 

it was reported that most states will monitor 
Chapter 2 expenditures for materials in terms of 
simple cost accounting procedures and programs in 
t£rms of counts of students served and staff involved. 

SEA evaluation directors foresee the increase in 
importance of Chapter 2 evaluation and are 
communicating this to local administrators. 

SEA Chapter 1 Directors 

Program monitoring, the evaluation and reporting system 
(TIERS) and sustained effects studies are the major Chapter 1 
monitoring and evaluation elements. Monitoring typically 
involves "going to schools and making sure they've targeted 
students correctly, that they're reporting correctly [and] 
collecting appropriate data, have files* have parent 
involvement — all the things that are basic parts of the law 
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[must] have some type of documentation; 16 Three of the Chapter 1 
respondents reported monitoring as a major task they perform. 
Most of the other tasks they perform have to do with 
(1) technical assistance regarding topics such as student 
selection and testing, (2) evaluation consultation related to 
sustained effects studies, and (3) program improvement. 

While all states indicated that they would continue the use 
of the Title I Evaluation and Reporting System (TIERS) , they also 
indicated various modifications that they had made. For example, 
some no longer require project performance data; others do not 
require districts to report if testing is in or but of level. 
The general sentiment was expressed by one respondent as follows: 

We're strongly suggesting to districts that they 
come up with an evaluation program that suits their 
needs, as well as continuing their reporting system 
using TIERS. 

I have an idea down the line somebody's going to 
say "Hey, we have to have some kind of reporting 
system," So we want to be prepared for that — thus 
we're going both routes. 

The sustained effect study is one way to design an evaluation 
to suit local district needs. Four of the five SEA Chapter 1 
respondents indicated an increased emphasis on sustained effects 
studies. The other one reported that there are "a lot of good 
studies going on." 

Sustained effects studies are intended to verify that gains 
made by students between the pretest and posttest are sustained 
over a longer period of time. 

Thus, a followup measure after the initial 
pre test— posttest cycle is needed. How soon after the 
posttest the information should be collected depends 
on what questions you want to examine. 

^f^^ievement gains occurring during the school 
year maintained over the_ summer months? 

2. Do the effects of Title I instruction continue 
after the students leave the program? 
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3. How are students performing who were in Title I 
last year and continued-in the Title I program ? 

(Demaline & Rader, n.d., pp. 8-9) 

In talking about his state's approach to sustained effects 
studies* one respondent explained: 



We are letting school districts know that it is 
one of the elements specified in the law and they 
must have a sustained effects study on file in 
the school district some time within the next 

three years. Starting with Chapter 1, some 

districts [are] documenting things for the first 
time that they assumed were taken care of by 
sending reports to the state office. 



State Chapter I respondents are generally unsure of the 
impact of the new legislation. They are Worried about cuts in 
program staff and the resulting loss of service to stude ts. 
They feel that the "number of students that need to be served and 
the number of services that need to be delivered has not 
changed." On the positive side, at least one respondent felt 
that the changes "got us out of the complacency we were 
in... people are taking a look at programs and trying to make them 
more effective." We will have to watch carefully the results of 
the monitoring visits and sustained effects studies* as well as 
those from TIERS. They will tell us the actual effects on 
students of the budget cuts and program changes . 



Summary of SEA Chapter 1 director preliminary findings . 

SEA Chapter 1 respondents reported monitoring and 
technical assistance as their major Chapter 1 tasks 
and mutual planning, leadership, and instructional 
support as the focus of their relationship with LEA 
Chapter 1 staff. 



An increased emphasis on sustained effects studies was 
also reported by SEA Chapter i directors. 
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Local Evaluation Directors 

A major task performed by most local evaluation units is the 
operation of the citywide testing program. They select tests, 
oversee the administration of the tests, arid analyze arid report 
the test data. Within this context they are also aware of the 
Chapter 1 testing program. There was agreement that TIERS would 
be continued. There were also some reports of interesting 
sustained effects studies going on that have potential 
district-wide implications. 

One city school system has been using an analytical procedure 
called Linear Structural Relations to develop causal models of 
classroom interactions. The purpose is to identify teaching 
behaviors which are related arid, in fact* causal to learning. 
They are in their third year of field trials of this study and 
expect to begin staff development sessions this year, based on 
the findings. 

In another school system, an extensive ethnographic 'Study of 
the teacher practices and administrative practices in Chapter 1 
school has been going on for the past two years. They are 
focusing on administrative styles* school climate, staffing 
problems, etc. During this year they will provide inservice 
training to their staff in ethnographic techniques. And, they 
are beginning to develop plans to address the problems that the 
study has thus far revealed. 

Typically, the Chapter i director does the administrative 
work for the program, including monitoring. The evaluation 
director and staff generally do not have any responsibility for 
this program. 

As reported in November interviews, the evaluation of 
Chapter 2 will vary, depending on the type of activity* e.g., 
program or materials, and whether the activity previously had an 
evaluation component tied to it. If the activity is still 
operating, then whatever evaluation accompanied it will 
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continue. If the activity is new and is a program , the following 
types of evaluation activities are likely to be conducted: 

• 1. keeping a number count of students as Well as an 
accomplishment count in terms of children in that 
program 

2. hiring a third party evaluatdr 

3. evaluating in terms of the objectives established 
by the school district (for example, in relation 
to the desegregation program: "as long as those 
schools maintain an ethnic balance along those 
guidelines, then we're saying that the funds are 
contributing to maintaining racial balance; if 
they attract and hold the right proportion of 
students, then what they're doing must be 
working") 



4. assessing educational impact on basic skills 

5. requiring an evaluation component as part of the 
mini-grant application process 

If the activity is new and is either equipment or materials , 
then accountabilty monitoring or simply recording what they're 
doing, (e.g., staff, expenditure of funds) is the most likely 
approach to be taken. Several LEAS noted that there is a 
shifting from evaluation to monitoring, as funding dries up and 
monies are allocated to tangible activities, e.g., materials. A 
loosening up of evaluation requirements from the SEA has also 
been reflected at the LEA level. 



Summary of LEA evaluation directors and Chapter 2 respondent 
preliminary findings . 

Analyzing district-wide test data is the major task of 
most LEA evaluation units. 

LEA evaluation directors reported innovative research 
and evaluation activities involving: 

-an ethnographic study of school practices and 
learning 

-a causal modeling study of teaching behaviors and 
outcomes 

-the adaptation of the evaluation components of 

previous categorical programs 
-the development of locally run mini-grant programs 
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Loca ± Chapter I D ixec t or s 

The tasks most often reported by Chapter 1 respondents were 
(i) assisting evaluation committees* principals* teacher s, etc. 
in doing needs assessments, (2) helping tiiem write measurable 
objectives for their applications, (3} providing technical 
assistance in interpreting and presenting their test results, 
(4) correlating the building programs with the district Chapter 1 . 
program and (5) involvement with specific programs like 
microcomputer labs, diagnostic/prescriptive programs, and reading 
resource models. 

Chapter 1 directors responded in the following numbers to a 
question about the focus of their interaction with building staff: 

4 technical assistance 

3 leadership 

2 monitoring 

1 supervision 

1 consultation 

1 support 

From their comments, it appears as though monitoring and 
technical assistance blend together pretty well. For example, 
one respondent described the relationship with school building 
staff as follows: 

[it's] more like technical assistance and 
consultation for [the] purpose of shaping up 
local evaluations and helping them understand 
their data* 

Another respondent in explaining what was meant by support said: 

w ® J 3o _ I ? OI1 i tc ? j: 9??_k5iidings; _?!*i-s consists °^ 
looking at student selection reports, student 
selection procedures, looking at 
curriculum. . .The n our specialist works with them 
in specific areas, e.g., reading and math. 

Finally, one respondent noted, "I monitor before a school is 
monitored by federal or state people. I come out and make sure 
that everything is in tiptop shape. I did this in the past and 
will continue it in the future." 
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When asked about the continuation of TIERS, three of the five 
indicated that it was about the same as before. The others 
listed such changes as tending to use percentiles rather than . 
Normal Curve Equivalents (NCEs) , and eliminating such information 
as comparability and project description data* One respondent 
expressed an interest in using the Instructional Accomplishments 
Information (IAI) System developed at Southwest Regional 
Laboratory to determine achievement. 

In discussing sustained effects studies, four of the five 
local Chapter 1 directors said that they will continue to do 
studies as they had done in the past. One said that at present 
they currently had no plans for a sustained effects study, but 
"it's one of the things that the district is aware that we have 
to provide. The evaluation unit will be responsible for getting 
data from schools, analyzing the data, and generating reports..." 

This ongoing commitment to sustained effects studies may be 
unique to large school districts. Perhaps more generalizable are 
the comments by the SEA Chapter 1 directory to the effect that, 
"starting with Chapter 1,' some districts [are] documenting things 
for the first time..." 

In general* then, local Chapter i directors have not seen 
much of an effect as far as any changes resulting from ECIA 
Chapter 1. The exceptions are that, because of reductions in' 
funding, they are serving fewer children, and that they are 
proceeding with sustained effects studies where these have not 
formerly been done. 



Summary of LEA Chapter 1 director preliminary findings . 

LEA Chapter 1 directors and their staff typically 
provide leadership in setting up district Chapter 1 
evaluation committees, give support through 
consultation and inservice programs, and supervise and 
monitor building programs to insure their 
compatability with the district program. 

Most large school districts were reported to have 
Chapter 1 sustained effects studies under way. 

LE7* Chapter 1 directors reported a relaxation in the 
testing and reporting requirements of Chapter 1. 
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Emerging Evaluation Information and Technical 
Assistance Needs 



Educational program evaluation has on the one hand been dealt 
a serious blow by the lack of evaluation requirements in the 
Education Consolidation and improvement Act of 1981. On the 
other hand/ a unique opportunity is present for this field of 
study and practice to show its true value. 

The Final Report and Recommendations of the Council of Chief 
State School Officers Consolidation Evaluation Task Force (May 
27, 1982) summarizes the situation well: 

With the continued reduced funding/ particularly for 
Chapter 2, the lack of federal requirements and 
guidelines for information collection and evaluation/ and 
the strong role foreseen for the state advisory 
committees/ the rationale for expending significant 
resources for evaluation may appear questionable. 

Yet the task force feels that the lack of evaluation data 
could very easily damage prospects for the future of the 
program. As the shift continues from the federal to the 
state levels in the management of education programs/ the 
states become more, not less, accountable for them. SEAs 
and LEAs have become accustomed to the federal government 
not only requiring the evaluation of programs but also 
dictating methods of evaluation. Since the federal 
regulations have been lifted it may be a temptation for 
SEAs and LEAs to lessen, evaluation activities. This 
could result only in a lack of information for policy 
decisions, an image of irresponsibility Which SEAs and 
LEAs do not deserve. The task force, therefore, strongly 
encourages each state to confront the evaluation 
questions emerging from the ECIA. 

(pp. 2-3) 

What are some of the important evaluation questions emerging from 
ECIA and what are their evaluation training and technical 
assistance implications? 

The following sets of questions are based on the summaries of 
findings which appear at the end of each section of this report. 
The related training arid technical assistance topics are 
alternatives to the standard approaches that might be used to 
address these questions. These new topics have been developed by 
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the Research on Evaluation Program. Their inclusion here is 
meant primarily to get reactions regarding their relevance to the 
evaluation questions. 

The questions and topics are just a "starter set." We are 
asking for additional questions arid topics as part of the review 
of this report by oar study respondents and other researchers. 
The purpose is to help us continue to monitor the impact of ECIA 
on state and local evaluation beyond the life of this study. 

The evaluation questions (E.Q.) arid their related training 
and technical assistance (T. & T.A.) topics listed next are 
grouped under the headings of policy, practice, and outcome. 

Policy 



E.Q.: How might future reductions in 

C^?P? e ?_VChapter 2 budgets be handled in 
order to maintain the quality of service 
provided? 

T. & T.A. : Multiple alternative modeling in 

determining fiscal rollbacks during 
educational funding crises. 

E.Q. : What are the anticipated state level 

costs and benefits of maintaining the 
Title I Evaluation arid Reporting system? 

T. & T.A. : Cost/benefit analysis. 

E.Q.: What state-wide and district-wide 

policies need to be set in order to 
insure the adequate evaluation of 
Chapter 2 programs? 

T. & T.A.: Document analysis and policy formulation. 

E.Q. : What state and local policies are needed 

to insure the appropriate and equitable 
purchase and use of microcomputers? 

T. & T.A.: Value analysis and policy formulation. 

E.Q. : To what extent are parents and others 

adequately included in the Chapter 1 
and/br Chapter 2 process? 

T. & T.A.: Document analysis, interview and survey 
techniques. 
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E.Q.: 

T. & T.A. 

Practice 



E.Q. : 

T. & T.A. : 
E.Q. : 

T. & T.A. : 
E.Q. : 

T. & T.A. : 
E.Q. : 

T. & T.A.: 
E.Q.: 

T. & T.A. : 

Outcomes 
E.Q. : 

T. & T.A.: 



What paperwork is legally necessary to 
meet Chapter 1 federal and state mandates? 

Legislative history and document analysis. 



What form should Chapter I supervision, 
monitoring, technical assistance, and 
leadership take in order to assure 
program quality? 

Conceptual analysis of Chapter 1 program 
quality, supervision, and the other 
concepts. The subsequent development of 
practical guidelines. 



What is the actual pattern of Chapter 2 
allocations? 

Public data source analysis. 

What are concrete examples of new 
practices which exemplify flexibility and 
decentralization resulting from Chapter 1 
and/or Chapter 2? 

Use of ethnographic, photographic, and 
other qualitative data collection 
techniques. 



How have local Chapter 1 programs changed 
as a result of reduced funding and 
decreased staff? 



Methods for evaluating program 
implementation. 

What evaluation related relationships are 
emerging between SEA and LEA, and between 
district and building staff in the 
implementation of Chapter 2 programs? 

Organizational development methods. 



What is the impact on students of various 
patterns of Chapter 2 allocations? 

Management consulting approach to program 
evaluation. 
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E.Q.: What impact have changes in flexibility 

and decentralization had on 
administrative burden in regard to 
Chapter 1 and/or Chapter 2? 

T. & T.A.: Document analysis and cost-analysis; 

E.Q.: What impact have local Chapter 1 staff 

reductions had on 

(1) the kinds of students served? 

(2) the educational achievement of 
various groups of students? 



T. & T.A. : State and local public data sources, 
quantitative re-analyses. 

E.Q.: Will the state and national TIERS 

outcomes be different as a result of 
differences in the academic achievement 
of students served by shrunken Chapter 1 
programs? 

T. & T.A.: State and national public data sources, 
quantitative re-analysis. 



E.Q.: How might the various local Chapter 2 

sustained effects studies be aggregated 
to produce valid and reliable 
generalizations? 

T. & T.A.: Meta-analysis. 
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FOOTNOTE 



For interested readers, the American Institutes, for Research 
document is a_ thorough textual comparison of Title I and 
Chapter 1 including applicable sections of the Senate and 
House of Representatives Conference Report. 
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APPENDIX ft 



Activities Under Chapter 2 

Subchapter A - Basic Skills Development 

This subchapter grants funds to develop and 
implement comprehensive and coordinated programs 
to improve elementary and secondary instruction in 
the basic skills of reading, mathematics, and 
written/oral communications. 

Allowable activities: 



1. School-wide diagnostic assessment. _ 

2. Establishment of learning goals and objectives. 

3. Pre-service and in-service training. 

4. Parent involvement and training. 

5. Student testing, program evaluation. 

6. pistrictwide activities, i.e., learning centers. 

7. Other activities to improve basic skills. 

Subchapter B - Educational Improvement and Support Services 
Allowable activities: 

1. Acquire and utilize school library resources, 
textbooks and other printed and published 
materials for use by teachers and students for 
instructional purposes. 

2. Instructional equipment and materials. 

3. Programs to improve LEA practices, particularly 
with activities designed to address educational 
problems such as the education of children with 
special heeds, i.e., educationally deprived, or 
gifted and talented children. 

4. Addressing problems of minority isolation and 
concentration, excluding busing. 

5. Guidance, counseling and testing programs, 
including those designed to help prepare students 
for employment. 

6. Programs and projects to improve the planning. 



management, and implementation of educational 
programs, including fiscal management. 



I. Teacher training and inservice staff development. 
8, Desegregation assistance. 

Subchapter C - Special Projects 

1 0 Metric education 

2„ Arts education 

3. Parent in-school partnership 

4. pre-school parent partnership 

5. Consumer education 
S 0 Youth employment 

7. Career education 

8. Environmental education 

9. Health education 

10. Law- re la ted education 

II. Population education 

12. Youth correction education 

13. Biomedical and medical sciences education 

14. Community schools 

15. Gifted and talented education 

16. Establishment of educational proficiency standards 

17. Safe schools/vandalism 

18. Ethnic heritage studies 
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APPENDIX B 



Chapter 1 Application Assurances 



The application described in subsection (a) shall be 
approved if it provides assurances satisfactory to the state 
educational agency that the local educational agency will keep 
such records and provide such information to the state 
educational agency as may be required for fiscal audit and 
program evaluation (consistent with the responsibilities of the 
?^? te _ a 5 e ?9?_? n ^ e ? their chapter) , and that the programs and 
projects described — 

(1) (A) are conducted in attendance areas of such 

agency having the highest concentration 
of low-income children; 

(B) are located in all attendance areas of an 
agency which has a uniformly high 
concentration of such children? or 

(C) are designed to utilize part of the 
available funds for services which 
promise to provide significant help for 
all such children served by such agency; 

(2) are based upon an annual assessment of educational 
heeds which identifies educationally deprived 
clildren in all eligible attendance areas* permits 
selection of those children who have the greatest 
need for special assistance, and determines the needs 
of participating children with sufficient specificity 
to ensure concentration on those needs; 

(3) are of sufficient size, scope, and quality to give 
reasonable promise of substantial progress toward 
meeting the special educational needs of the children 
being served and are designed and implemented in 
consultation with parents and teachers of such 
children; 

(4) will be evaluated in terms of their effectiveness in 
achieving the goals set for them, and that such 
evaluations shall include objective measurements of 
educational achievement in basic skills and a 
determination of whether improved performance is 
sustained over a period of more than one year; and 

(5) make provision for services to educationally 
deprived children attending private elementary, and 
secondary schools in accordance with section 557. 

(PL 97-35, August 13, 1982, p. 95 STAT. 465) 
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